Parallelization Techniques for Sparse Matrix Applications1

information to fetch data and to perform the computation. Such runtime preprocessing techniques have been fairly well-studied and successfully incorporated into compilers [14, 6]. The inspector/executor method incurs a runtime overhead for each inspector stage which are greatly increased when references use multiple levels of indirection. To reduce these overheads in sparse-matrix problems, the sparse-array rolling (SAR) technique was proposed in [15]. This method uses special representations for distributed sparse matrices which allows efficient access to the nonzero elements. Initial evaluations of the SAR method were promising and led to its integration into the Vienna Fortran compiler using language extensions. For a detailed description of the compilation issues, we refer the reader to [15]. In this paper, we focus on the SAR runtime support and describe how it allows efficient access to the distributed sparse matrix. The performance of SAR depends on a wide range of different parameters such as the data distribution, the locality characteristics of the algorithm, and the sparsity of the input matrix. We have evaluated its performance under many combinations of these choices, and we describe the impact that each choice has on preprocessing costs, executor efficiency, and memory overhead. The paper is structured as follows. Section 2 provides a general overview of sparse matrix parallelization. Section 3 provides an overview of the SAR technique and its runtime support. Section 4 introduces a sparse-matrix program and illustrates how it is parallelized using the SAR approach. We discuss the impact of distribution choices in Section 5. Section 6 provides a detailed performance evaluation. The last two sections discuss related work and the conclusions drawn from this work.